Referring to FIG. 1, an interline charge coupled device (CCD) image sensor 10 is comprised of an array of photodiodes 20. The photodiodes are covered by color filters to allow only a narrow band of light wavelengths to generate charge in the photodiodes. Typically image sensors have a filter with a pattern of three or more different colors arranged spanning over the photodiodes in a 2×2 sub array as shown in FIG. 2. For the purpose of a generalized discussion, the 2×2 array is assumed to have four colors, A, B, C, and D. The most common color filter pattern used in digital cameras is often referred to as the Bayer pattern: color A is red, colors B and C are green, and color D is blue.
Referring back to FIG. 1, image readout of the photo-generated charge begins with the transfer of some or all of the photodiode charge to the vertical CCD (VCCD) 30. In the case of a progressive scan CCD, every photodiode simultaneously transfers charge to the VCCD 30. In the case of a two field interlaced CCD, first the even numbered photodiode rows transfer charge to the VCCD 30 for first field image readout, then the odd numbered photodiode rows transfer charge to the VCCD 30 for second field image readout.
Charge in the VCCD 30 is read out by transferring all columns in parallel one row at a time into the horizontal CCD (HCCD) 40. The HCCD 40 then serially transfers charge to an output amplifier 50.
FIG. 1 shows an array of only 24 pixels. Many digital cameras for still photography employ image sensors having millions of pixels. A 10-megapixel image sensor would require at least ⅓ second to read out at a 40 MHz data rate. This is not suitable if the same camera is to be used for recording video. A video recorder requires an image read out in 1/30 second. The shortcoming to be addressed by the present invention is how to use an image sensor with more than 1 million pixels as both a high quality digital still camera and 30 frames/second video camera.
The prior art addresses this problem by providing a video image at a reduced resolution (typically 640×480 pixels). For example, an image sensor with 3200×2400 pixels would have only every fifth pixel read out as described in U.S. Pat. No. 6,342,921. This is often referred to as sub-sampling, or sometimes as thinned out mode or skipping mode. The disadvantage of sub-sampling the image by a factor of 5 is only 4% of the photodiodes are used. A sub-sampled image suffers from reduced photosensitivity and alias artifacts. If a sharp line focused on the image sensor is only on the un-sampled pixels, the line will not be reproduced in the video image. Other sub-sampling schemes are described in U.S. Pat. Nos. 5,668,597 and 5,828,406.
Prior art including U.S. Pat. No. 6,661,451 or U.S. patent application publication No. 2002/0135689 A1 attempt to resolve the problems of sub-sampling by summing pixels together. However, this prior art still leaves some pixels un-sampled.
U.S. patent application publication No. 2001/0010554 A1 increases the frame rate by summing pixels together without sub-sampling. However, it requires a two field interlaced read out. It is more desirable to obtain a video image with progressive scan read out. Interlaced video acquires the two fields at different times. A moving object in the image will appear in different locations when each interlaced field is acquired.
Another disadvantage of the prior art is it only reduces the image resolution in the vertical direction. In the horizontal direction, the HCCD must still read out every pixel. Only reducing the image resolution through sub-sampling or other methods in the vertical direction does not increase the frame rate to 30 frames/second for very large (greater than 8 million pixels) image sensors.
U.S. patent application publication No. 2003/0067550 A1 reduces the image resolution vertically and horizontally for even faster image readout. However, this prior art requires a striped color filter pattern (a 3×1 color filter array), which is generally acknowledged to be inferior to the Bayer or 2×2 color filter array patterns.
If an image sensor has a resolution of 5120×3840 pixels then a factor of five resolution reduction would produce an image matching the XVGA video resolution standard of 1024×768 pixels. The prior art U.S. Pat. No. 6,342,921 provides a method of a 5× resolution reduction through sub-sampling. Instead of sub-sampling, a method for a 5× resolution reduction is needed that samples all pixels of the image sensor.
In view of the deficiencies of the prior art, an invention is desired which is able to produce 30 frames/second video from a megapixel image sensor with a 2×2 color filter pattern while sampling 100% of the pixel array and reading out the video image progressive scan (non-interlaced).